Field of the Invention
This invention is related to optical three dimensional (3D) measurement techniques, and more particularly to methods of generating and displaying multiple-surface 3D images.
Description of the Related Art
The ability to measure a 3D object accurately and to render a 3D image of it on a two-dimensional (2D) display is very important to a variety of academic and industrial applications. For example, a bio-chip used for DNA sequencing may contain thousands of minute wells partially filled with reagent and sealed with a thin layer of plastic cover. Because reagent is expensive, it is important to measure the fill level inside a well so that reagent waste can be eliminated. The challenge in this case is to accurately profile the many surfaces involved, namely the surface of plastic cover, the top surface of the well, the fluid surface of the reagent, and the bottom surface of the well, and to display the 3D structure and measurement results in a way that is easy to understand.
Over the years, several types of optical based non-destructive measurement systems have been developed to address the aforementioned applications. These systems are typically based on techniques such as confocal microscopy and structured light sources (SLS).
For example, U.S. Pat. No. 4,198,571 issued to Sheppard in 1980 discloses the basic technique of confocal microscopy. U.S. Pat. No. 5,022,743 (Kino) discloses an improved confocal system using a Nipkow disk. U.S. Pat. No. 5,065,008 (Hakamata) describes a confocal system based on laser scanning. U.S. Pat. No. 6,838,650 (Toh) describes an improved high speed and high resolution confocal system for three dimensional measurement. U.S. Pat. No. 7,372,985 (So) discloses a confocal based system and method for volumetric 3D displaying of biological tissue.
Systems based on structured light sources (SLS) offer similar capability to that provided by confocal microscopy. For example, U.S. Pat. No. 7,729,049 (Xu) describes a 3D microscope using one of various SLS techniques.
Both confocal microscopy and SLS 3D measurement systems generate a 3D image by capturing multiple 2D images at a set of Z steps within a Z scan range. In the case of a confocal system, an algorithm based on maximum image intensity is used to determine a surface. In an SLS system, such as the one disclosed in U.S. Pat. No. 7,729,049 (Xu), maximum image contrast is used instead. Because both confocal and SLS systems can image through optically translucent materials, any interior surfaces inside a transparent object can, in principle, be measured. For example, U.S. Pat. No. 7,323,705 (Haga) discloses a method and apparatus to measure liquid volume of small bio-cells by measuring the top surface of the liquid and the bottom of the well. However, Haga only measures an average value for each of the surfaces, and not the 3D profile of the surfaces. U.S. Pat. No. 7,227,630 (Zavislan) discloses a confocal system that can produce vertical sections of a sample by displaying the various internal parts using image intensity values, but does not create an image in the form of extracted surfaces.
In practice, profiling an interior surface of a transparent object is not trivial. For example, it is difficult for an optical 3D measuring system to find the boundary surface between two liquids with similar optical properties. Furthermore, a transparent object could add aberration to the system optics to produce undesirable artifacts. Because of these difficulties, internal surfaces extracted by a conventional 3D measuring system often contain false surfaces and demonstrate various degrees of image distortion. Without effective means for separating between false and valid surfaces, a conventional 3D system will not be able to present the true internal structure of an object.
U.S. Pat. No. 7,372,985 (So) discloses a system combining confocal optics with direct volumetric rendering for imaging tissue samples. The volumetric data generated by this system is a collection of pixel values at a regular XYZ grid obtained by stacking a set of sequentially captured 2D images. In the direct volumetric rendering scheme, stacked 2D image pixel values are directly mapped into 3D. Various segmentation methods have also been suggested to enhance the volume image by separating valid image from noise.
Instead of direct volumetric rendering, U.S. Pat. No. 6,556,199 (Fang) discloses a method and apparatus to convert intensity volume data set into a voxel-based volume representation. A voxel-based (i.e. a volumetric pixel-based) volumetric display of 3D images has been available as a computer infrastructure is well known. For example, U.S. Pat. No. 6,940,507 (Repin) discloses such a volume rendering process with fast rendering time and improved visual image quality. Compared to direct volumetric rendering, voxel-based volumetric rendering offer flexibility in displaying the volume data. For example, it can extract internal surfaces of an object as well as display 2D pixel values in 3D. While a voxel based 3D measurement system may be ideal for viewing biological specimens with many irregular internal parts, it is not optimized for industrial parts that have well-defined internal structures.
For an industrial part, such as a micro-fluidic circuit or a bio-chip, it is highly desirable to view all of its surfaces in one 3D image and to profile these surfaces in precision. The 3D systems that offer direct volumetric rendering do not generate surfaces, so they are of limited utility. Systems that use voxel-based volumetric rendering are not optimized for industrial parts due to their limited surface extraction precision, slow speed of rendering, and lack of interactive surface selection.
Therefore, a need arises for a technique to generate accurate 3D image of an object, to render the object in multi-surface 3D view, and to measure various parameters of these surfaces.